What is the Space of Camera Response Functions?

Many vision applications require precise measurement of scene radiance. The function relating scene radiance to image brightness is called the camera response. We analyze the properties that all camera responses share. This allows us to find the constraints that any response function must satisfy. These constraints determine the theoretical space of all possible camera responses. We have collected a diverse database of real-world camera response functions (DoRF). Using this database we show that real-world responses occupy a small part of the theoretical space of all possible responses. We combine the constraints from our theoretical space with the data from DoRF to create a low-parameter Empirical Model of Response (EMoR). This response model allows us to accurately interpolate the complete response function of a camera from a small number of measurements obtained using a standard chart. We also show that the model can be used to accurately estimate the camera response from images of an arbitrary scene taken using different exposures. The DoRF database and the EMoR model can be downloaded at http://www.cs.columbia.edu/CAVE. 1 Scene Radiance to Image Brightness Researchers in computer vision develop algorithms to determine scene properties like shape, reflectance, and illumination from images. Many of these algorithms require precise measurements of scene radiance to recover the scene properties. Examples of algorithms that explicitly use scene radiance measurements are color constancy [7, 13], construction of linear high dynamic range images [17, 4, 16], photometric stereo [2, 18, 20], shape from shading [12], estimation of reflectance and illumination from shape and brightness [14], recovery of BRDF from images [5], and surface reconstruction using Helmholtz stereopsis [21]. What connects scene radiance with image brightness? The optics of the imaging system gather light rays from scene points and focus the rays on the image plane [11]. An electronic or chemical photo-detector converts image irradiance to image brightness. ∗This work was completed with support from a National Science Foundation ITR Award (IIS-00-85864) and a grant from the Human ID Program: Flexible Imaging Over a Wide Range of Distances Award No. N000-14-00-1-0929 1To simplify our exposition, we include integration time in irThe goal of this work is to provide an accurate and convenient model of the mapping from scene radiance to image brightness. In general, this mapping comprises several complex factors, such as vignetting, lens fall-off, the sensitivity of the detector, and the electronics of the camera [1, 10]. Regardless of the individual factors involved, we can assume the mapping is a composite of just two functions, s and f , as shown in Fig. 1. The function s models the effect of transmission through the optics of the system. It may vary spatially over the image but is generally linear with scene radiance [1]. The function f models the process by which the irradiance E of an image point is converted to an image brightness B. This f is generally a non-linear function of image irradiance and is called the camera response function. In many imaging devices, the non-linearity of f is intentional. A non-linear mapping is a simple means to compress a wide range of irradiance values within a fixed range of measurable image brightness values. Manufacturers produce photographic films with specific nonlinear characteristics. The responses of digital cameras are often designed to mimic the non-linearities of film. Though non-linear, a camera’s response function is generally uniform across the spatial dimensions of the image. Hence, it is described by a one-variable function of irradiance, B = f(E). Inversion of the camera response function allows the transformation of image brightness to image irradiance. Going from image irradiance to scene radiance can then be accomplished by finding s, which is easy to do once f is known [1, 10]. Therefore, we will focus our attention on the response function f . A number of algorithms have been introduced in computer vision and computer graphics to estimate the camera response f from multiple images of a scene taken with different exposures [4, 15, 16, 17, 19]. All these methods make a priori assumptions about the form of the response function. For example, by assuming the response has the form of a gamma curve, f(E) = α + βE , Mann and Picard [16] find the parameters γ, α, and β from multiple registered images of a static scene taken using different exposures. Mann also proposed other analytic forms for the response [15]. radiance E. Thus, E = tẼ, where t is the integration time and Ẽ is the irradiance per unit time. 2It was recently shown in [8] that, to avoid ambiguities, a priori constraints on the response function are imperative when finding the response from multiple images. Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’03) 1063-6919/03 $17.00 © 2003 IEEE